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(57) Abstract 

A user controlled combination video game apparatus (102) and exercise system. The system includes video game apparatus for 
displaying an interactive video game (1 10) requiring user participation. The interactive video game includes a playing piece. The system 
also includes hand activity sensor apparatus (114) for providing a signal associated with the independent and unconstrained movement of 
each of a user's hands for determining at least one aspect of the playing piece within said interactive video game and a playing piece. The 
system can also include feet activity sensor apparatus (116) for providing a signal associated with an activity executed by the user's feet, 
and torso activity sensor apparatus (1 18) for providing a signal associated with an activity executed by the user's torso for determining at 
least one aspect of the playing piece within the interactive video game. 
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A USER CONTROLLED COMBINATION VIDEO GAME 
AND EXERCISE SYSTEM 

FIELD AND BACKGROUND OF THE INVENTION 

The present invention relates to user controlled combination video 
5 game and exercise systems in general and in particular to those user 
controlled combination video game and exercise systems which require 
both effort and time-space coordination on behalf of the user. 

Various patents have attempted to integrate video game apparatus 
with exercise devices for encouraging users to maintain a regular exercise 
10 program by providing fun, challenging and self-motivating environments. 

For example, U.S. Patent 4,278,095 to Lapeyre describes a treadmill 
and a monitor for displaying the user's speed and distance and a variable 
speed outdoor exercising scene controlled by the speed of the user on the 
treadmill. However, the exercising scene is not interactive per se in that 
1 5 the user does not participate in a competitive interactive video game. 

While U.S. Patent 4,925,189 to Braeunig describes a body mounted 
video game controller which attaches to the user's upper body allowing the 
user to play a video game by leaning the upper body in different directions 
to simulate the movement of a joystick. The user may play from a 
20 standing, sitting, kneeling position and therefore this user controlled 
combination video game and exercise devices suffers from the disadvantage 
that the user is not required to engage in intense physical activity during 
a training session. 

And while U.S. Patent 4,720,789 to Hector et al. describes a video 
25 exercise or game floor controller with location indicating foot pads which 
allow an operator to input information into the system by locating his feet 
on foot pads of the floor controller. The game floor controller can also 
include light segments associated with each of the foot pads to encourage 
rapid foot movement by the fast switching on and off of different light 
30 segments. However, the apparatus does not measure whether the user is 
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merely shifting his weight from one foot to the other foot in a tap dancing 
style or whether the user is sustaining intense physical activity, for 
example, by running on the spot. 

Other developments include adjusting the video game difficulty level 

5 upon the effort executed by the user during exercise. Typically the 
difficulty level of the video game is regulated such that the heartbeat of the 
user remains within a pre-determined specified target zone. The exercise 
is executed on an exerciser such as a treadmill, standing bicycle, standing 
rowing machine, and the like including speed sensors for indicating the 

10 effort executed by the user. Examples of such exercisers are described in 
U.S. Patents 4,489,938 to Darzinskis, 4,512,567 to Phillips, 4,542,897 to 
Melton et al., 4,637,605 to Ritchie, 4,660,828 to Weiss, 4,709,917 to Yang, 
4,711.447 to Mansfield, 4,817,950 to Goo, 5,001,632 to Hall-Tipping, 
5,054.771 to Mansfield, 5,139,261 to Openiano, 5,148,152 to Stuekle et aL, 

15 5,195,746 to Boyd et al. and 5,277,678 to Friedabach and 5,308,296 to 
Eckstein. 

The existing devices evaluate physical effort either by monitoring 
the device itself or by the monitoring the heartbeat of the user. The 
monitoring of the device itself suffers from the disadvantage that the 

20 aerobic effort expended in activities which are not performed directly on 
the device cannot be monitored. The disadvantage of monitoring the user*s 
heartbeat is that the heartbeat is a subjective measurements which depends 
on the condition and fitness of the specific user and does not constitute an 
objective determination which can be used to compare a number of users 

25 as in an actual sports game. 

Therefore, there is a need for a user controlled combination video 
game and exercise device while overcoming the deficiencies of the above- 
mentioned apparatus. More specifically, there is a need for a device which 
monitors both the activity of the user and an objective physical parameter 

30 which does not depend on the condition and fitness of the user and which, 
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as a result, makes it possible to evaluate activities which cannot be 
properly evaluated heretofore, such as, for example, running in place, 
jumping, knee lifts in the sitting position, and the like. 

SUMMARY OF THE INVENTION 
5 The present invention is for a user controlled combination video 
game and exercise system which encourages the user to engage in intense 
aerobic activity during a challenging and motivating training session. In 
addition, the system can be employed for user participation in a wide range 
of recreational activities, such as dancing, and the like. 

10 Hence, there is provided according to the present invention, a user 

controlled combination video game apparatus and exercise system, the 
system comprising: (a) video game apparatus for displaying an interactive 
video game requiring user participation, the interactive video game 
including a playing piece; (b) limb activity sensor apparatus for providing 

15 a signal associated with the independent and unconstrained movement of 
at least two of a user's limbs for determining at least one aspect of the 
playing piece within the interactive video game; and (c) a playing piece 
controller responsive to the signal for controlling the playing piece. 

According to a further feature of the present invention, the limb 

20 activity sensor apparatus includes a left hand activity sensor operable by 
the left hand of the user and a right hand activity sensor operable by the 
right hand of the user. 

According to a still further feature of the present invention, the limb 
sensor apparatus includes a left foot activity sensor operable by the left 

25 foot of the user and a right foot activity sensor operable by the right foot 
of the user. 

According to a yet still further feature of the present invention, at 
least a portion of the limb activity sensor apparatus is worn by the user. 
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According to a yet still further feature of the present invention, at 
least a portion of the limb activity sensor apparatus is held by the user. 

According to a yet still further feature of the present invention, at 
least a portion of the limb activity sensor apparatus is remote from the 
5 user. 

According to a yet still further feature of the present invention, at 
least a portion of the limb activity sensor apparatus is associated with an 
exercise apparatus on which the user exercises. 

According to a yet still further feature of the present invention, the 
10 exercise apparatus provides information associated with an activity 
executed by the user. 

According to a yet still further feature of the present invention, the 
limb activity sensor apparatus includes an actuator button. 

According to a yet still further feature of the present invention, the 
15 playing piece controller controls one aspect of the playing piece selected 
from the following group: the number of pixels moved by the playing piece 
per screen refresh, the size of the playing piece, the shape of the playing 
piece, the color of the playing piece, the location of the playing piece, the 
direction of movement of the playing piece, the firing rate of the playing 
20 piece, and the interaction of the playing piece with other objects on the 
screen. 

According to a yet still further feature of the present invention, the 
interactive video game is played between the user and another user. 

According to a yet still further feature of the present invention, the 
25 system further comprising torso activity sensor apparatus for providing a 
signal associated with an activity executed by the user's torso for 
determining at least one aspect of the playing piece within the interactive 
video game. 

According to a yet still further feature of the present invention, at 
30 least a portion of the torso activity sensor apparatus is worn by the user. 
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According to a yet still further feature of the present invention, at 
least a portion of the torso activity sensor apparatus is associated with an 
exercise apparatus on which the user exercises. 

According to a yet still further feature of the present invention, the 
5 system further comprising a heartbeat detector apparatus for providing a 
signal associated with an activity executed by the user for determining at 
least one aspect of the playing piece within the interactive video game. 

According to a yet still further feature of the present invention, the 
system further comprising a system controller responsive to the signal for 
10 controlling at least a portion of the system. 

According to a yet still further feature of the present invention, the 
system controller controls at least one feature of an exercise apparatus. 

According to a yet still further feature of the present invention, the 
system further comprising strain sensors for detecting the force applied by 
15 at least one limb of the user. 

There is also provided apparatus for measuring aerobic activity 
executed by a user, the apparatus comprising: (a) at least one sensor for 
detecting the impact frequency of at least one of the user's feet on a 
surface; and (b) at least one sensor for detecting the percentage of time that 
20 at least one of the user's feet is on a surface. 

There is still further provided a user controlled combination video 
game apparatus and exercise system, the system comprising: (a) video 
game apparatus for displaying an interactive video game requiring user 
participation, the interactive video game including a playing piece; (b) 
25 activity sensor apparatus for providing a signal associated with the 
ambulatory speed of a user for determining at least one aspect of the 
playing piece within the interactive video game; and (c) a playing piece 
controller responsive to the signal for controlling the playing piece. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The invention is herein described, by way of example only, with 
reference to the accompanying drawings, wherein: 

FIG. 1 shows a block diagram of the preferred embodiment of a user 
5 controlled combination video game and exercise system constructed and 
operative according to the teachings of the present invention; 

FIGS. 2a and 2b depict a Squash video game screen and a Squash 
video game instruction screen, respectively, according to the teachings of 
the present invention; 
10 FIGS. 2c and 2d depict a Survival Training video game screen and 

a Survival Training video game instruction screen, respectively, according 
to the teachings of the present invention; 

FIG. 2e depicts a Badminton Training video game screen according 
to the teachings of the present invention; 
15 FIG. 2f depicts a Dance Training video game screen according to 

the teachings of the present invention; 

FIGS. 3a-3d depicts several representative system control screens 
provided by the video game apparatus of the system including a Start 
Training Session screen, a Select Video Game screen, a Select Difficulty 
20 Level screen and a Select Training Duration screen, respectively; 

FIG. 4 shows a schematic illustration of a first embodiment of the 
user controlled combination video game and exercise system of Figure 1 
in which the aerobic activity sensor apparatus includes transmitters worn 
by the user and receivers integrated with the video game apparatus of the 
25 system; 

FIG. 5 shows a schematic illustration of the first embodiment of the 
user controlled combination video game and exercise system of Figure 1 
being used by two users concurrently; 

FIG. 6 shows a schematic illustration of a second embodiment of the 
30 user controlled combination video game and exercise system of Figure 1 
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in which the aerobic activity sensor apparatus includes transmitters worn 
by the user and receivers integrated with an exercise apparatus configured 
as an exercise mat; 

FIG. 7 shows a schematic illustration of a third embodiment of the 
5 user controlled combination video game and exercise system of Figure 1 
in which the aerobic activity sensor apparatus includes feet sensors and 
hand activity sensors integrated with an exercise apparatus configured as 
an exercise mat; 

FIG. 8 shows a schematic illustration of a fourth embodiment of the 
10 user controlled combination video game and exercise system of Figure 7 
in which the exercise mat is rotatably mounted on a base; 

FIGS. 9a and 9b show a schematic illustration of a fifth embodiment 
of the user controlled combination video game and exercise svstem 
including a treadmill; 
15 FIGS. lOa-lOc show three operative positions of the hand activity 

sensors for detecting three heights of one of the user's hand; 

FIGS. 11a and lib show schematic illustrations of hand activity 
sensors for detecting direction of movement of a user's hands along one 
axis and along two axes, respectively; 
20 FIG. 12 shows a table entitled "Speed of User's Playing Piece as a 

Function of the Level of Aerobic Activity of the User as measured by Foot 
Sensors"; and 

FIG. 13 shows a table entitled "Speed of User's Playing Piece as a 
Function of the Hand Speed of the User as Measured by Hand Sensors". 

25 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The present invention is of a user controlled combination video 

game and exercise system. 

The principles and operation of the user controlled combination 

video game and exercise system of the present invention may be better 
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understood with reference to the drawings and the accompanying 
description. 

With reference now to the drawings, Figure 1 shows a block 
diagram of a user controlled combination video game and exercise system, 
5 generally designated 100, constructed and operative according to the 
teachings of the present invention. Generally speaking, user controlled 
combination video game and exercise system 100 of the present invention 
marries the intense concentration and total involvement required by a user 
in playing a challenging interactive video game with exercise. However, 
10 at the same time, system 100 can be employed for non-strenuous 
recreational activities, for example, for instructing dancing, and the like. 
In both cases, system 100 requires skilful co-ordination in the manner in 
which the user executes an activity in response to events taking place 
within the video game. 
15 Hence, system 100 includes video game apparatus 102 including a 

video game controller 104 for displaying an interactive video game on a 
display 106 observable by a user. Display 106 is preferably a conventional 
TV monitor and therefore the connection between video game apparatus 
102 and display 106 is achieved in a similar fashion as between a video 
20 player and a TV monitor. Alternatively, display 106 can be a dedicated 
monitor, a projected screen, a hologram and similar display system. Video 
game apparatus 102 also includes a system controller 108 for controlling 
the operation of user controlled combination video game and exercise 
svstem 100. Svstem controller 108 controls such operations as initiating 
25 a training session, selecting a video game, selecting a game difficulty, 
selecting the duration of a training session, providing a summary of 
pertinent information regarding the performance of the user, and the like, 
in addition, system controller 108 can be used for regulating a feature of 
an exercise apparatus, for example, the resistance of a treadmill. 
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Video game apparatus 102 is preferably designed to accept 
cartridges 110 wherein each cartridge 110 provides interactive video games 
of a particular type. The video games can also be classified according to 
other criteria, for example, number of users. The number of users can be 
5 one, two or more. In the case of a single user, the user plays against video 
game apparatus 102 while in the case of two or more users, the users can 
play either against each other or against video game apparatus 102. 

The interactive video games are typically competitive sport-type 
games, arcade-type games, Gameboy-type games, and the like for 

10 encouraging the user to engage in intense aerobic activity and to maintain 
an intensive training schedule. Sport-type games can include racket games, 
for example, squash, tennis, ball games, for example, football, basketball, 
and other games such as boxing, adventures, indoor activities, outdoor 
activities. Arcade-type games can include, for example, Tetrus*. Pacman™, 

15 etc. Furthermore, the competitive interactive video games can include 
queries, for example, mathematical problems, logic problems, and the like 
which the user is required to answer during a session on system 100 such 
that the user is challenged both on the physical level and on the cognitive 
level. 

20 The competitive interactive video games of video game controller 

104 include one or more playing pieces under the control of a playing 
piece controller 112. The playing pieces depend on the nature of the video 
game in question. Typical playing pieces include, but are not limited to, 
a racket in a racket game, a player in a ball game, a "monster" in a 

25 Pacman™ game, and the like. Also, the playing piece can be hidden within 
in the video game, the whole screen, and the whole scenario. In principle, 
playing piece controller 112 controls the playing piece in such a fashion 
that the user is more likely to achieve a higher score, a victory or similar 
desirable game outcome through increasing his level of effort. However, 

30 as will become apparent below, it is not the mere level of effort which 



WO 96/05766 



PCT/US95/I0694 



10 

secures a desirable game outcome but also the skilful time and space co- 
ordination, and response to queries, if any, as practiced by the user while 
participating during the ongoing video game as displayed on display 106. 
However, it should be noted that the term "interactive video game" 
5 also refers to a wide range of low aerobic intensity simulation scenarios, 
animation scenarios, virtual reality scenarios, and the like as known in the 
art. For example, a simulation scenario can include a training session for 
teaching how to dance a dance, how to perform judo exercises, gymnastic 
activities, and the like. In the case of teaching dancing, the user can be 
10 represented as one member of a dancing couple while video game 
apparatus 102 provides the second member of the dancing couple. Hence, 
the member of the dancing couple representing the user is the playing piece 
under the control of playing piece controller 110. Alternatively, two users 
can dance together. In both cases, the desirable game outcome can be a 
15 score similar to that given in conventional dancing competitions. As 
before, it can be readily appreciated that the interactive video game 
outcome requires skilful time and space co-ordination on behalf of the user. 

System controller 108 and playing piece controller 112 receive input 
from one or more sources including hand activity sensor apparatus 114, 
20 feet activity sensor apparatus 116, and torso activity sensor apparatus 118. 
Generally speaking, hand activity sensor apparatus 114 detects independent 
and unconstrained movement of each of the user's arms. While feet 
activitv sensor apparatus 116 detects independent and unconstrained 
movement of each of the user's legs. And torso activity sensor apparatus 
25 118 detects any independent and unconstrained movement of the user's 
torso. It can be readily appreciated that system 100 can include other 
activity sensors for detecting activity of other parts of the body, for 
example, the head, knees, elbows, etc. 

It is a particular feature of the present invention that within the 
30 context of the present invention, independent movement implies that the 
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limbs of the user can move independently of each other, i.e. the left arm 
can be moved upwards while the right arm remains stationary, rather then 
being constrained to execute a common motion or an opposing motion, for 
example, when gripping handlebars of a bicycle, manipulating oars of a 

5 rowing boat, and the like. It is a further feature of the present invention 
that within the context of the present invention unconstrained movement 
implies that the limbs of the user can be moved in any natural movement 
rather then being constrained to execute a specific movement, for example, 
turning the handlebars of a bicycle, running along a treadmill, riding a 

10 bicycle, and the like. 

Hand activity sensor apparatus 114, feet activity sensor apparatus 
116, and torso activity sensor apparatus 118 typically each include a Look- 
Up-Table having a "game value" for the manner in which a particular 
activity is executed. The manner in which "game values" are awarded for 

15 movements demanding a greater expenditure of aerobic energy or greater 
space-time co-ordination depend on the type of video game. For example, 
in a tennis video game, a greater expenditure of aerobic energy can be used 
for increasing the speed of the racket of the user. While, in a Pacman 
video game, a greater expenditure of energy can be used for slowing down 

20 a "monster" trying to catch the user. And while in a dance teaching video 
game, a more difficult step can be give a greater "game value". 

The likelihood of achieving a higher score, a victory or other 
desirable game outcome is correlated to the activity executed by the user 
through adapting one or more features of the playing piece. First, playing 

25 piece controller 112 can control the maneuverability of a playing piece 
such that its maneuverability is a function of the level of aerobic activity 
of the user or the level of time-space coordination displayed by the user. 
Maneuverability typically refers to the "speed" and "location" of the 
playing piece given in terms of the number of pixels moved within each 

30 screen refresh of display 106. Second, playing piece controller 112 can 
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control the size of a playing piece such that its size is a function of the 
level of aerobic activity of the user or the level of time-space coordination 
displayed by the user. Third, playing piece controller 112 can control the 
"firing rate" of a playing piece such that the firing rate is a function of the 
5 level of aerobic activity of the user or the level of time-space coordination 
displayed by the user. Fourth, playing piece controller 112 can control the 
shape of the playing piece such that its shape is a function of the level of 
aerobic activity of the user or the level of time-space coordination 
displayed by the user. Fifth, playing piece controller 112 can control the 
10 color of the playing piece such that its color is a function of the level of 
aerobic activity of the user or the level of time-space coordination 
displayed by the user. And sixth, playing piece controller 112 can control 
the interaction of the playing piece with other objects in a video game such 
that its interaction is a function of the level of aerobic activity of the user 
1 5 or the level of time-space coordination displayed by the user. 

Furthermore, system controller 108 and playing piece controller 112 
can also receive input from a heartbeat detector apparatus 120. Heartbeat 
detector apparatus 120 can be employed, for example, for terminating a 
training session when the heartbeat of a user exceeds a pre-determined 
20 value according to his age, general state of fitness, etc. Alternatively, 
heartbeat detector apparatus 120 can be employed instead of hand activity 
sensor apparatus 114, feet activity sensor apparatus 116, or torso activity 
sensor apparatus 118 for providing input regarding the level of aerobic 
activity executed by a user. Still further, system controller 108 and playing 
25 piece controller 112 can also receive input from actuator buttons 121 
employed on hand activity sensor apparatus 114, feet activity sensor 
apparatus 116, or torso activity sensor apparatus 118 as known in the art 
of joystick controllers and the like. 
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With reference now to Figures 2a-2f, the screens depict 
representative interactive video games playable by one or more users on 
user controlled video game and exercise system 100. 

Turning now to Figures 2a and 2b, Figure 2a depicts a Squash video 
5 game screen 122 while Figure 2b shows depicts a Squash video game 
instruction screen 124. The purpose of the Squash video game is to keep 
a ball 126 within a court 128 as long as possible. This is achieved by the 
user moving a racket 130 located at the bottom of the screen to the left and 
to the right such that ball 126 rebounds off racket 130, thereby remaining 

10 in court 128. The number of times that the user successfully intercepts ball 
126 is shown on a scoreboard 132. In this case, the user is required to run 
faster to move racket 130 faster and to raise either his left or right hand to 
move racket 130 to the left or the right, respectively. Hence, it can be 
readily appreciated that the user has to display considerable bodily 

15 coordination and exert considerable aerobic activity in order to achieve a 
high score. 

Turning now to Figures 2c and 2d, Figure 2c depicts a Survival 
Training video game screen 134 while Figure 2d shows depicts a Survival 
Training video came instruction screen 136. The purpose of the Survival 

20 Training video game is to clear a way in a jungle 138 to reach food shown 
as apples 140, to avoid being eaten by monsters 142 shown as squares and 
to kill one or more monsters 142 within a 3 second period after having 
eaten an apple 140. This is achieved by the user moving an "action man- 
playing piece 144 along the pathways through jungle 138. The number of 

25 apples 140 that the user eats and the number of monsters 142 that he kills 
is shown on a scoreboard 146. In this case, the user is required to run 
faster to move playing piece 144 faster, to raise his left or right hand to 
move playing piece 144 to the left or the right, respectively, raise both 
arms to move playing piece 144 up and to lower both arms to move 

30 playing piece 144 down. Again, it can be readily appreciated that the user 
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has to display considerable bodily coordination and exert considerable 
aerobic activity in order to achieve a high score. 

Turning now to Figure 2e, Figure 2e depicts a Badminton Training 
video game screen 148. The purpose of the Badminton Training video 
5 game is to keep a ball 150 within a court 152 as long as possible. This is 
achieved by the user moving a racket 154 located at the bottom of the 
screen to the left and to the right such that ball 150 rebounds off racket 
154, thereby remaining in court 152. The number of times that the user 
successfully intercepts ball 150 is shown on a scoreboard 156. In this case, 
10 the user is required to raise either his left or right hand to move racket 154 
to the left or the right, respectively. By running faster, the user can 
increase the size of racket 154. In another version, the speed that the user 
runs determines the height of rebound of ball 150 from racket 154 such 
that the height of rebound increases with a faster running speed. Hence, 
15 it can be readily appreciated that the user has to display considerable 
bodily coordination and exert considerable aerobic activity in order to 
achieve a high score. 

Turning now to Figure 2f, Figure 2f depicts a Dance Training video 
game screen 158. The purpose of the Dance Training video game is to 
20 teach a user a dance. This is achieved by the user being represented a first 
member 160 of a dancing couple under the control of playing piece 
controller 110 while video game apparatus 102 provides a second member 
162 of the dancing couple. A score is provided according to the manner 
in which the user performs the dance routine correctly. As before, it can 
25 be readily appreciated that the interactive video game outcome requires 
skilful time and space co-ordination on behalf of the user to achieve a high 
score. 

With reference now to Figures 3a-3d, and as described with 
reference to several representative system control screens, hand activity 
30 sensor apparatus 114, feet activity sensor apparatus 116, torso activity 
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sensor apparatus 118 and heartbeat detector apparatus 120 can provide 
input to system controller 108 for controlling the operation of user 
controlled combination video game and exercise system 100. In particular 
Figure 3a depicts a Start Training Session screen 164 in which the user is 

S required to stand on an exercise apparatus and raise his hands to initiate a 
training session. While Figure 3b depicts a Select Video Game screen 166 
in which the user is required to raise one hand to scroll through video 
games options and to raise both hands to select a particular video game 
that he wants to play. Figure 3c depicts a Select Difficulty Level screen 

10 168 operative in a similar manner to the Select Video Game screen 166 
except that the user is selecting a game difficulty, for example, Novice, 
Beginner, Intermediate and Advanced. And finally, Figure 3d depicts a 
Select Training Duration screen 170 operative in a similar manner to the 
Select Video Game screen 166 except that the user is selecting the duration 

15 of a training session. 

With reference now to Figures 4-9, the schematic illustrations depict 
several implementations of user controlled combination video game and 
exercise system 100 constructed and operative according to the teachings 
of the present invention. 

20 Turning now to Figure 4, hand activity sensor apparatus 114, feet 

activity sensor apparatus 116 and torso activity sensor apparatus 118 are 
realized as transmitters and receivers couplings communicating through an 
IR interface, an RF interface and the like. The transmitters are typically 
worn by the user and the receivers are typically integrated with video game 

25 apparatus 102. Hence, the user can execute independent and unconstrained 
movements of his arms and legs as long as communication between the 
transmitters and the receivers is maintained. In the case of an IR interface, 
communication is maintained along the line of sight while in the case of 
an RF interface communication is maintained within the range of the 

30 transminer-receiv°r coupling. Typically, each transmitter emits signals 
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having a particular characteristic such that the signals of each transmitter 
can be distinguished from the signals of the other transmitters. The 
receivers are deployed in such a manner that they can determine the 
location of each of transmitters and typically sample each transmitter in 
5 sequence. 

Hence, hand activity sensor apparatus 114 includes two hand sensors 
172 and 174 for attaching to the user's wrists for detecting movement of 
the user's hands, feet activity sensor apparatus 116 includes two feet 
sensors 176 and 178 for attaching to the user's ankles for detecting 
10 movement of the user's feet, torso activity sensor apparatus 118 includes 
a waist sensor 180 for attaching to the user's waist for detecting movement 
of the user's torso and video game apparatus 102 includes three receivers 
182-186. Obviously, depending on the movement in question, some 
movements are registered by only one sensor, for example, raising the right 
15 arm is only detected by sensor 172 while other movements, for example, 
jumping in the air, is registered by all of sensors 172-180. In addition, 
heart beat counter 120 provides the heart beat of the user during an 
exercise session on system 100. 

It can be readily appreciated that playing piece controller 112 is 
20 required to compute relative movements of one sensor compared to another 
sensor to determine the "game value" of some movements. For example, 
playing piece controller 112 only requires information from sensor 172 for 
determining the game value of an activity such as the user raising his hand 
from waist height to a raised height above his head while he is standing 
25 stationary. While, in the case, that playing piece controller 112 is required 
to determine the game value of an activity such as the user raising and 
lowering his hands from waist height to the same raised height while 
jumping up in the air, playing piece controller 112 requires information 
from sensors 172, 174 and 180. This is because the absolute movements 
30 of sensors 172 and 174 are relative to the movement of sensor J 80. 



WO 96/05766 



PCTAJS95/10694 



17 

Turning briefly to Figure 5, system 100 is depicted for the 
concurrent exercising of two users acting as adversaries in a common 
interactive game. In this case, the transmitters denoted a transmit signals 
at different frequencies from those transmitters denoted b such that system 
5 100 can distinguish between the movements of the two users. 

In the implementation of system 100 depicted in Figure 6, the 
signals of sensors 172-180 are picked up by receivers 182-186 embedded 
in an exercise mat 188. Again, receivers 182-186 are deployed such that 
playing piece controller 112 can determine the displacement of each sensor 
10 172-180. In this case, system 100 includes a communication link 190 
between receivers 182-186 and video game apparatus 102 realized as a 
flexible, electric cable, an IR interface, an RF interface, and the like. 
Alternatively, exercise mat 188 can be provided as a stepper or as a stair- 
type configuration. 

15 Exercise mat 188 is similar in many respects to conventional 

exercise mats in that it is fabricated from a plastic covered, heaw duty 
sponge substrate designed for cushioning shocks derived from executing 
intense aerobic activity, for example, running on the spot. Also, exercise 
mat 188 preferably of sufficient size such that the user can perform 

20 intensive aerobic exercise without having to pay undue attention to his 
position thereon so as not to constrain the independent and unconstrained 
movement of the user's legs. 

In the implementation of system 100 depicted in Figure 7, hand 
activity sensor apparatus 114 includes hand sensors 192 and 194 and feet 

25 activity sensor apparatus 116 includes feet sensors 196 and 198, both 
realized as pressure sensitive devices, load cells, switches, and the like 
integrated in an exercise mat 200. As shown, hand sensor 192 is deployed 
to the right of exercise mat 200 for detecting the movement of the user's 
right hand and hand sensor 194 is deployed to the left of exercise mat 200 

30 for detecting the mov ement of the user's left hand. Hand activity sensor 
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192 is actuated by an elastic cord 202 provided with a handle 204 for 
gripping by the user's right hand while hand sensor 194 is actuated by an 
elastic cord 206 provided with a handle 208 for gripping by the user's left 
hand so as not to constrain the independent and unconstrained movement 
5 of the user's arms. In a similar manner, foot sensor 196 is located 
substantially at the center of the right half of exercise mat 200 for detecting 
the impact of the user's right foot on exercise mat 200 while foot sensor 
198 is located substantially at the center of the left half of exercise mat 200 
for detecting the impact of the user's left foot on exercise mat 200. As 
10 before, communication link 190 between hand sensors 192 and 194 and 
feet sensors 196 and 198 and video control apparatus 102 is through a 
flexible, electric cable, an IR interface, an RF interface, and the like. To 
extend the range of activities which can be executed on exercise mat 200 
and the muscles which can be exercised, modifications to exercise mat 200 
15 can include rotatably mounting it on a base 210 as shown in Figure 8. In 
this case, system 100 preferably includes a hand support 211 including 
actuator buttons 121. 

With reference now to Figure 9a and 9b. system 100 is shown 
including a treadmill 212 which includes a speed sensor 213 for measuring 
20 the ambulatory speed of the user on treadmill 212. It should be noted that 
ambulatory speed within the context of the present invention refers to any 
speed from a gentle walking speed to a fast running speed. In this case, 
speed sensor 213 replaces hand activity sensor apparatus 114, foot activity 
sensor apparatus 116 and torso activity sensor apparatus 118 as input to 
25 system controller 108 and playing piece controller 112. As mentioned 
hereinabove, system controller 108 can regulate one or more features of an 
exercise apparatus, for example, the resistance of treadmill 212. Again, 
system 100 preferably includes a hand support 211 including actuator 
buttons 121. As before, the user participates in an interactive active video 
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game which requires both physical exertion and time and space 
coordination to achieve a desirable game output. 

With reference now to Figures 10a- 10c, the principle by which hand 
sensors 192 and 194 measures three heights, the speed and the acceleration 
5 of the user's hands is now described. In this case, hand sensor 192 
includes two normally open (NO) switches 214 and 216 and a connector 
218 extending therebetween. It should be noted that cord 202 is not 
attached to the center of connector 218 but rather towards one of switches 
214 and 216 such that switches 214 and 216 do not close concurrently. 

10 Hence, the movement of the user's right hand is determined by whether 
none, one or both of switches 214 and 216 are closed from their normally 
open states. In this case, cord 202 is connected closer to switch 214 such 
that switch 214 closes before switch 216 regardless of the direction of 
movement of the user's right hand. Turning now to Figure 10a, it can be 

15 seen that when the user holds handle 204 at waist height, then both 
switches 214 and 216 are open. While as shown in Figure 10b, when the 
user holds handle 204 at shoulder height, then switch 214 is closed and 
switch 216 is open. And finally, as shown in Figure 10c ? it can be seen 
that when the user holds handle 204 at a raised height above his head, then 

20 both switches 214 and 216 are closed. The speed and acceleration of the 
user's hands can be derived from the time taken to move through the three 
different states. 

With reference now to Figures 11a and lib, it can be readily 
appreciated that hand sensors 192 and 194 can also be designed to detect 
25 the direction of movement along one axis or both axes in Cartesian 
coordinates. For detecting direction of movement along one axis, hand 
sensors 192 and 194 require two sensor units 220 and 222. While, for 
detecting direction of movement along two axes, hand sensors 192 and 194 
require three sensor units 224, 226 and 228. 
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With reference now to Figures 12 and 13, the tables depict typical 
examples of the manner in which playing piece controller 112 correlates 
one or more features of a playing piece with the user's level of aerobic 
activity such that the likelihood of achieving a higher score, a victory or 
5 other desirable game outcome increases with an increased level of aerobic 
activity. For the sake of exposition, the tables are used by playing piece 
controller 112 when the user is playing the Squash video game on the 
implementation of system 100 shown in Figure 7. Hence in this case the 
playing piece is the racket. 
10 It is a particular feature of the present invention that feet activity 

sensor apparatus 116 determines the level of aerobic activity by taking into 
account two factors: the impact frequency f of the user's feet on a surface 
and the airborne percentage of time p of the user's feet defined as the 
percentage of time that both feet are airborne. It can be readily appreciated 
1 5 that it is not sufficient to take into account just the impact frequency by 
considering the amount of aerobic activity required when gently hopping 
from one foot to the other foot at a certain impact frequency as compared 
to when jogging on the spot at the same frequency as compared to running 
on the spot at the same frequency. 
20 However, because of the cyclic nature of walking or running, it is 

typically sufficient to take into consideration the impact frequency and the 
airborne percentage of only one foot. Furthermore, the impact frequency 
can be taken of one foot while the airborne percentage can be taken of the 
other foot. Still further, the percentage of time that one or both of the 
25 user's feet on a surface can be measured rather than measuring the airborne 
percentage as described above as they are complements of one another. 
The surface can be a floor or alternatively an exercise apparatus, for 
example, exercise mat 200. 

Turning now to Figure 12, the table depicts a 5 x 6 matrix of 30 
30 game scores for different levels of aerobic activity wherein the impact 
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frequency f is divided into five frequency levels: f < 1, 1=1 or 2, f = 3, 4 
or 5, f = 6 or 7 and f > 8 and the airborne percentage p is divided into six 
levels: "Heavy Walking" for when p < - 40%, "Walking" for when p is 
between -40% to 0%, "Jogging" for when p is between 0% to 24%, 

5 "Running" for when p is between 24% to 50%, "Fast Running" for when 
p is between 50% to 70% and lastly "Sprinting" for when p is > 70%. It 
should be noted that when p takes on a negative value, this implies that 
both of the user's feet are on the ground. As shown, at the lower levels 
of aerobic activity, for example, f = 1 or 2 and Heavy Walking, the speed 

10 of the racket is 0 pixels per screen refresh i.e. the racket does not move. 
In a similar fashion, at the higher levels of aerobic activity, for example, 
f = 6 or 7 and Fast Running, the speed of the racket is seven pixels per 
screen refresh. 

As described above with reference to Figure 2b, the user raises his 
1 5 hands for moving the racket in the left and right directions. Hand activity 
sensor apparatus 114 can also provide a game value as to the hand speed 
sensed by hand sensors 192 and 194 such that the raising of the hands 
provides an additional input to the Squash video game. In this case, hand 
activity sensor apparatus 114 is responsive to the rate by which handles 
20 204 and 208 are raised by the user for awarding him "bonus pixels" in 
terms of the movement of the racket. 

The speed by which say handle 204 is raised can be determined by 
the delay time d between actuations of switches 214 and 216 (see Figures 
10a- 10c). With reference now to Figure 13, the delay time d is preferably 
25 divided into five levels: d > 200 msecs, d is between 160 to 200 msecs, d 
is between 120 to 160 msecs, d is between 90 to 120 msecs and lastly d 
is less than 90 msecs. The bonus pixels are provided as a function of the 
number of screen refreshes, in the example, five screen refreshes such that 
a total of twenty-five conditions are represented. As shown, for example, 
30 when d is greater than 200 msecs, the user receives no bonus pixels over 
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the next five screen refreshes while when, for example, d is less than 90 
msecs, the user receives 4 bonus pixels in the first screen refresh, 3 bonus 
pixels in the second screen refresh, and so on. 

While the invention has been described with respect to a limited 
5 number of embodiments, it will be appreciated that many variations, 
modifications and other applications of the invention may be made. 
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WHAT IS CLAIMED IS: 

1. A user controlled combination video game and exercise 
system comprising: 

(a) video game apparatus for displaying an interactive video 
game requiring user participation, said interactive video game 
including a playing piece; 

(b) feet activity sensor apparatus providing a signal determining 
frequency of a cyclical aerobic activity and level of difficulty 
of said aerobic activity; and 

(c) a playing piece controller responsive to said signal for 
controlling said playing piece within said interactive video 
game. 

2. The system of claim 1, wherein said frequency of a cyclical 
aerobic activity is determined by the impact frequency of at least one of 
the user's feet on a floor. 

3. The system of claim 1 , wherein said level of difficulty of said 
aerobic activity is determined by the percentage of time of least one of the 
user's feet on a floor. 

4. The system as in claim 1 . wherein said feet activity sensor 
apparatus provides a signal determining the impact frequency of each of 
the user's feet on said exercise surface and the percentage of time of each 
of the user's feet on a floor. 

5. The system as in claim 1, wherein said feet activity sensor 
apparatus includes an exercise surface for providing said signal. 
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6. The system as in claim 1, further comprising hand activity 
sensor apparatus for monitoring a three dimensional working space, said 
apparatus providing a signal determining the location of at least one of the 
user's hands at any location within said working place, said playing piece 
controller responsive to said signal for controlling said playing piece within 
said interactive video game. 

7. The system as in claim 5, further comprising hand activity 
sensor apparatus for monitoring a three dimensional working space, said 
apparatus providing a signal determining the location of at least one of the 
user's hands at any location within said working place, said playing piece 
controller responsive to said signal for controlling said playing piece within 
said interactive video game. 

8. The system as in claim 6, wherein at least one portion of said 
hand activity sensor apparatus is worn by the user, said at least a portion 
of limb activity sensor communicating with said playing piece controller 
by wireless means. 

9. The system as in claim 6 ; wherein at least a portion of said 
hand activity sensor apparatus is held by the user, said at least portion of 
limb activity sensor communicating with said playing piece controller by 
wireless means. 

10. The system as in claim 6, wherein at least a portion of said 
hand activity sensor apparatus is remote from the user, said at least a 
portion of hand activity sensor communicating with said playing piece 
controller by wireless means. 
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11. The system as in claim 1, further comprising a second feet 
activity sensor apparatus for providing a signal associated with the impact 
frequency of each of a second user's feet on a second exercise surface and 
the percentage of time that each of the second user's feet is on said second 
exercise surface such that said interactive video game is playable between 
a first user and the second user. 

12. A user controlled combination video game and exercise 
system comprising: 

(a) video game apparatus for displaying an interactive video 
game requiring user participation, said interactive video game 
including a playing piece; 

(b) hand activity sensor apparatus providing at least one signal 
determining the distance of at least one of the user's hands 
from a pre-determined reference point within a three- 
dimensional training space; and 

-(c) a playing piece controller responsive to said at least one 
signal for controlling said playing piece within said 
interactive video game. 

13. The system as in claim 12, wherein at least a portion of said 
hand activity sensor apparatus is worn by the user, said at least portion of 
hand activity sensor communicating with said playing piece controller by 
wireless means. 

14. The system as in claim 12, wherein at least a portion of said 
hand activity sensor apparatus is held by the user, said at least a portion of 
hand activity sensor communicating with said playing piece controller by 
wireless means. 
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15. The system as in claim 12, wherein at least a portion of said 
hand activity sensor apparatus is remote from the user, said at least a 
portion of limb activity sensor communicating with said playing piece 
controller by wireless means. 

16. The system as in claim 12, further comprising an exercise 
surface providing a signal associated with the impact frequency of each of 
the user's feet on said exercise surface and the percentage of time that the 
user's feet is on said exercise surface. 

17. The system as in claim 16, wherein at least a portion of said 
hand activity sensor apparatus is held by the user. 

18. The system as in claim 12, further comprising a second hand 
activity sensor apparatus providing at least one signal determining the 
distance of at least one of a second user's hand from a pre-determined 
reference point within a three-dimensional training space such that said 
interactive video game is playable between a first user and the second user. 
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